A network criterion for the success of cooperation in an evolutionary prisoner's 
dilemma, and a variation on Hamilton's rule 



OS 

o 
o 

(N 

bJQ! 

< 



43 

i 

O ' 

o : 

o 

>v 
43 

Oh! 



> 

oo 
cn 

od 
O 

o 



X 



Stephen Devlin 1 and Thomas Treloar 2 

1 Mathematics Department, University of San Francisco, 
2130 Fulton Street, San Francisco, CA 94117, USA 
2 Mathematics Department, Hillsdale College, 33 E. College St., Hillsdale, MI, 49242, USA 

(Dated: August 26, 2009) 

We show that the success of cooperation in an evolutionary prisoner's dilemma on a complex 
network can be predicted by a simple, quantitative network analysis using mean field parameters. 
The criterion is shown to be accurate on a wide variety of networks with degree distributions 
ranging from regular to Poisson to scale-free. The network analysis uses a parameter that allows for 
comparisons of networks with both similar, and distinct, topologies. Furthermore, we establish the 
criterion here as a natural network analogue of Hamilton's classical rule for kin selection, despite 
arising in an entirely different context. The result is a network-based evolutionary rule for altruism 
that parallels Hamilton's genetic rule. 



INTRODUCTION 



Altruistic behavior among agents in evolving systems, 
both biological and social, has been widely observed in 

nature Q, M H H 0, IE 0- The fact that coopera- 
tive behavior can emerge between unrelated individu- 
als in the competitive landscape of natural selection, 
however, is paradoxical. Evolutionary game theory is 
widely employed to address this question, and models 
of evolutionary systems that can exhibit realistic phe- 
nomena are of great interest to researchers across disci- 
plines includingphysics, bi ology, a nd the social sciences 
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Complex networks have played a central role in the 
study of evolutionary systems [11, 12, 3]. In particular, 



network-based models, where evolution is driven by the 
discrete replicator dynamics, have been shown to support 
robust cooperation that is unsustainable in traditional 
models built on unstructured populations. In a network- 
based model, agents occupy the vertices of the network, 
and interact only within their immediate neighborhood 
consisting of those agents to whom they are connected 
by network edges. Interactions take the form of a mathe- 
matical game, often the prisoner's dilemma (PD), which 
captures in a precise framework the temptation to self- 
ishly advance one's own fitness at the expense of a co- 
operating neighbor. Evolution is implemented using the 
replicator dynamics. 

In [ll|, Nowak and May pioneered the network-based 
approach by showing that cooperation in the PD could 
become evolutionarily sustainable on a lattice. In [l3j |. 
Santos and Pacheco further showed that cooperation 
could even become the dominant population trait on cer- 
tain heterogeneous networks like those with scale-free de- 
gree distributions, where the network's vertices have de- 
grees that follow an inverse power law. A great deal of 
subsequent research has focused on the interplay between 
complex networks and evolutionary dynamics. 



In this paper we show that the equilibrium success of 
cooperation in an evolutionary PD on a network (defined 
precisely below) can be effectively predicted from a quan- 
titative network parameter derived using a mean field 
system analysis. Our approach is rooted in the ideas of 
generating functions associated to random networks [l^ | , 
and uses results from empirical studies of the dynam- 
ics. We compare our theoretical predictions to Monte- 
Carlo simulations and find excellent agreement across 
networks with varying topologies and varying average de- 
grees. Given the inherent complexity of these dynamical 
systems, the accuracy with which the simple criterion 
here predicts actual dynamics is especially appealing. 

Finally, we give an interpretation of the criterion de- 
rived here as an analogue of Hamilton's rule for kin selec- 
tion; a classical result in genetics that explains emerging 
cooperative behavior among related individuals. In do- 
ing so, we are able to apply techniques from the study of 
complex networks to a modeling problem in evolutionary 
biology, and arrive at a result that both brings insight to 
both the factors that drive the model, as well as the so- 
cial and biological questions that originally inspired the 
model. 



2. PRELIMINARIES 

The PD is widely studied as a framework in which to 
model problems involving conflict and cooperation. Two 
players choose between cooperation (C) and defection 
(D). Players' strategy choices determine (normalized) 
payoffs-interpreted as fitness in evolutionary biology-as 
follows: mutual cooperation gives R to each player, a de- 
fector exploiting a cooperator gets T, and an exploited 
cooperator gets S. Two defectors each give and receive 
nothing, and payoffs satisfy T > R > > S . A rational 
player will choose to defect since the payoffs for defec- 
tion strictly dominate those of cooperation regardless of 
the co-player's strategy. The result is a Nash equilibrium 
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where both players defect; the dilemma arising from the 
inefficiency of this equilibrium: both agents would fare 
better by cooperating Q. 

Define the cost c of cooperation in the PD to be the 
payoff forgone (from a defector's perspective) by choosing 
to cooperate, or c = T — R. Let b represent the benefit 
provided to a co-player by a cooperator, so that b — 
R — S. Thus, the cost-to-benefit ratio of the game is 



T-R 
R-S' 



(1) 



The cost-to-benefit ratio indicates of the temptation to 
defect inherent in the game, with larger r corresponding 
to a stronger temptation to defect. 

A widely adopted payoff normalization 11, 13|, 14. 15. 



161 . Il7l | sets R = 1 and S — 0, so that the game depends 
on the single parameter 1 + r indicating the temptation 
toward defection in the game. Taking S close to zero 
amounts to an assumption that social interactions are 
inexpensive. With this normalization, the game lies on 
the boundary between the PD and the snowdrift game 
(SG), another commonly studied game of cooperation. 
In the SG, the the bottom two PD game payoffs are re- 
versed so that cooperation is a better unilateral response 
to defection: T > R > S > P. In that case, and setting 
T = 1 + r, R = 1, and P = 0, the Nash equilibrium is 
1 — , so is close to zero as long as S is sufficiently close 
to zero. Qualitatively, the case of S — (the so-called 
weak PD) addresses both games when social interactions 
are inexpensive, so is the focus from here on. While the 
S fts assumption is both plausible and widely adopted, 
it is significant, and drop ping it has a considerable effect 
on system dynamics [20|. 

Evolution is introduced through repeated interactions 
between agents with respect to the replicator dynamics. 
The replicator dynamics model natural selection using 
agent fitness comparisons that result in stochastic im- 
itation of fitter strategies by less fit strategies (details 
below). In the repeated PD, payoffs are further required 
to satisfy T + S < 2R in order to ensure that full coop- 
eration in the population remains Pareto optimal. 

When a population of agents is unstructured and agent 
interactions are random, the replicator dynamics favor 
defection and cooperation is driven to extinction. As 
mentioned in the introduction, the situation is strikingly 
different when the population is structured by a network. 

Consider a network Af consisting of vertices and undi- 
rected edges where neither loops nor multiple edges are 
allowed. Agents occupy the network's vertices and are 
constrained to interact only with their immediate neigh- 
bors; those agents with whom they are connected by an 
edge. A round consists of each agent playing a pure strat- 
egy in a PD with all neighbors, and accumulating the 
resulting payoffs. Following a round of play, agents si- 
multaneously update strategies using discrete replicator 
dynamics: if agent x has accumulated payoff P x , and 



compares her payoff to that of agent y, then x will adopt 
the strategy of y with probability 



Pr 



max{0,P a - P x } 
(1 + r)fc max 



with /c max equal to the larger of the degrees of vertices x 
and y [IE EE ES E3, El ■ 

We perform simulations on various specific networks 
(details below) with 10 4 vertices, in each case starting 
from a random strategy assignment where the probability 
of an agent cooperating is 0.50. A series is defined to 
consist of 10 4 rounds of play and updating. The series 
mean is taken to be the average cooperation level over 
the last 1000 rounds of the series. For a given network, 
100 series are run, and the equilibrium cooperation level 
is taken to be the average of these 100 series means. 

It is well documented, and summarized below, that 
cooperation can become evolutionarily stable in network 
models of this kind. Moreover, the extent of the evo- 
lutionary success of cooperation has been shown to de- 
pend greatly on the particular network topology involved 
Ju, El, El- ln order to explore this phenomenon further, 
we recall some basic tools in the study of networks. 

Let pk denote the probability that a random vertex 
from the network J\f has degree k, and let X be the 
random variable that takes values in the set of possi- 
ble degrees of vertices in the network. The probability 
generating function El for the distribution of X is given 
by 



G(x) 



^PkX k , 

k>0 



and gives a first-order approximation of network topol- 
ogy. The degree distribution ignores any other contact 
information present in the network, so G(x) represents a 
generic network chosen randomly from among all those 
with the fixed degree distribution. The average vertex 
degree V in the network is given by V = G'(l) = (k). 

If an edge is randomly chosen from the network and 
followed to a vertex at one end, it is k times more likely 
to lead to a vertex of degree k than a vertex of degree 1. 
Therefore, if Y is the random variable whose values are 
the degrees of vertices reached along random edges, then 
the probability generating function of Y is 



T(x) = 



J2k>o kpkx k 

T,k>0 k Pk 



G'(l) ^ 

y ' k>0 



k PkX k = ^T77T- (2) 



G'(l) 



Define a random neighbor to be the vertex reached by 
first choosing a random vertex in the network, followed by 
a random edge emanating from that vertex. If no degree- 
degree correlations are present in the network, then it 
follows that T{x) is the probability generating function 
for the degree distribution of random neighbors. The 
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average degree of a random neighbor N is therefore the 
expected value of Y, so that TV = T'(l). Note that when 
the probability of an edge leading from a degree j vertex 
to a degree k vertex is not independent of j, then N 
need not equal T"(l) [2lj. The mean field parameters N 
and V, of course, require sufficiently large systems to be 
meaningful [22| . 

A critical factor emerging from studies of cooperation 
phenomena is network heterogeneity [HI, ED, H 3 13] • 
In heterogeneous networks, a broad diversity of vertex de- 
grees are represented. In the context of the evolutionary 
PD, network heterogeneity has been shown to be stro ngly 
correlated with increased success of cooperators 13j, LL7] . 
On certain scale-free networks, for example, cooperation 
can be the dominant population trait for the full range 
of PD game parameters. 

Heterogeneity can be naturally quantified by the vari- 
ance of the degree distribution. With (k) denoting the ex- 
pected value of the random variable X, and (k 2 ) denoting 
the expected value of X 2 , one has Var[X] = (k 2 ) — (k) 2 . 
Using the notation above, 

Var[X] = G'(1)T'(1) - G'(l) 2 = V(N - V). 

If we fix the average network degree V, then (N — V), or 
the size difference between an average neighbor and an 
average vertex in the network, dictates network hetero- 
geneity. 

Since cooperation thrives on heterogeneous networks, 
(N — V) emerges as a critical network parameter and has 
been studied in 17, [2lj. In the following, we introduce 
a framework that explains this fact in the context of the 
specifics of the evolutionary PD. 



2.1. Previous Results 

In the evolutionary PD, payoffs flow through connec- 
tions to cooperators, and agents benefit from maximizing 
their access thereto. It is well known that cooperation 
can thrive throu gh t he formation of clusters of cooper- 
ating agents 11, [H, 0|. Moreover, when social inter- 
actions are inexpensive (S close to zero]^ coop erators of 



large degree are especially stable [13|, lis, LL3] ■ By con- 
trast, as a large defector converts her neighborhood to 
defectors, she significantly reduces her own payoff and 
becomes susceptible to takeover by a cooperator. For 
this reason, larger degree vertices have been shown to 
disproportionately favor cooperation [Til ] . 

A more detailed picture of the dynamics emerges in 
[l5j |. For low temptation to defect, cooperation is the 
social norm. As the temptation to defect increases, the 
dynamics are governed by three populations: a core (or 
cores) of cooperating agents, a core (or cores) of defecting 
agents, and a critical fluctuating population of sometime- 
cooperators and sometime-defectors. The resilience of 



cooperators, as discussed in [15(, is determined by in- 
teractions between agents on the border of a cooperator 
core. When the temptation to defect is too high, de- 
fectors eventually invade the core by stripping off layer 
upon layer of exposed cooperators until they are largely 
eradicated from the population. 



3. RESULTS AND DISCUSSION 

Since the growth or breakdown of a cooperator core is 
determined by the core's exposure to fluctuating nodes, 
that is where we focus our analysis. Consider an interac- 
tion on the frontier of a cooperating cluster. Using the 
mean field network parameters N and V, we address the 
question of predicting the particular value of the cost-to- 
benefit ratio, call it ro.5, at which point neither a coop- 
erator nor a defector has an advantage (on average). At 
^0.50 one expects each strategy to be equally successful, 
and a resulting equilibrium where cooperation and de- 
fection are approximately equally prevalent. Thus, ro.50 
is a predicted threshold at which point the system tran- 
sitions between dominant (defined as more than 50%) 
cooperation and dominant defection. 

Given the description of the dynamics in [151 ] (and out- 
lined above), we consider interactions between agents on 
the boundary of a cluster of cooperators with the goal of 
deriving a criterion to calculate ro.50 ■ 

Of course, within a cooperator or defector core, no 
strategy changes occur as a cooperator (resp. defector) 
considers the success of a another cooperator (resp. de- 
fector). The dynamics are determined by interactions 
between differing strategists. 

Consider, therefore, a randomly chosen vertex and her 
randomly chosen neighbor. The two possible configura- 
tions are that of a cooperator comparing payoff with a 
defector and vice-versa. 

First we analyze a defecting vertex, representing a 
chain of potentially advancing defectors, and connected 
to a cooperating neighbor on the border of a core of coop- 
erators. Our assumptions give the defector, Dy, the av- 
erage network degree V, while her cooperating neighbor, 
Cn, has the average network neighbor size N. Finally, 
assume that Cn has k defecting neighbors while Dy has 
I defecting neighbors. 

We consider the relative costs and benefits associated 
to the strategies (as opposed to pure payoffs) in this situ- 
ation, with costs and benefits as in Eq. {]]). The cooper- 
ator perceives the collective value of defection in a round 
of play to be (V — l)b. The defector, meanwhile, sees the 
cooperator receiving a benefit b from each of her coop- 
erative neighbors, but also sees this benefit mitigated by 
the cost of each cooperative act. From the defector's per- 
spective, the value of cooperation is (N — k)(b — c) — kc. 
Fig. Q] shows the situation when k = I = 1, that is, when 
both agents are maximally cooperator connected, and 
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FIG. 1: The critical frontier between a defector chain (blue) 
and a cooperator cluster (red). The darker colored vertices 
are labeled with the respective costs and benefits associated to 
those strategies and collected over one round of play. Com- 
parison of these values in the context of strategy updating 
leads to the criterion of Eq. (|6]). 



thus have maximal strength in the sense of evolutionary 
fitness. 

Since ro.50 is the cost-to-benefit ratio at which point 
neither agent perceives an advantage in the other's strat- 
egy, we can predict ro.50 by equating the collective ben- 
efits of the D strategy and the C-strategy. This gives 

{V - l)b = {N - k)(b - c) - kc. 



Rearranging, we get 



c (k-l) 
b N 



N-V 
N 



(3) 



Substituting c = (T — R) and b = (R — S), and inserting 
the normalized payoffs T = l + r, R= 1, and S = gives 



k-l N-V 



N 



N 



(4) 



Finally, we add an assumption that near r.50, the term 
w 0. This amounts to an assumption that when an 
average cooperator and average defector are connected by 
an edge, their respective numbers of defecting neighbors 
are comparable. That this assumption is true in actual 
simulations is verified below. 

With the added assumption the condition simplifies to 



r.50 



N-V 
N ■ 



(5) 



Defectors will advance against cooperators when the 
equality becomes an inequality reflecting an inherent ad- 
vantage to defectors on the frontier. 

Conversely, consider a randomly chosen cooperator Cy 
on a frontier, compared to her randomly chosen neighbor 
No- As above, let I and k be the numbers of defector 
neighbors of Cy and No, respectively. The same argu- 
ment this time yields the condition 

V-N l-k 



V 



V 



Now, however, the condition is statistically unsustainable 
since (V — N) < 0, and | > 0. During the system's 
evolution, smaller cooperators not inside a cluster are 
stripped away. As a result, cooperators must migrate 
to larger vertices in order to survive, leaving the first 
scenario already analyzed above. 

This framework suggests the following: natural selec- 
tion favors cooperation when the cost-to-benefit ratio of 
the cooperative act is smaller than the relative size differ- 
ence between an average neighbor and an average vertex, 
divided by the average neighbor size: 



r < 



N-V 



N 



(6) 



The term 



N-V 
N 



which is purely a network parameter 



as well as a natural measure of network heterogeneity 
on a zero-to-one scale, should mark the phase transition 
from dominant cooperation (more than half the popula- 
tion cooperating) to dominant defection (more than half 
the population defecting). 

To test the accuracy of Eq. ([5]), we perform simulations 
on networks with various degree distributions. Networks 
with K vertices, and average degree 2m are constructed 
via a two step process introduced in [2311. F irst, a net- 
work is generated using the algorithm in [23j . This algo- 
rithm uses a single parameter a to in terp olate between an 
Erdos-Renyi random network (ER) [24| , and a Barabasi- 
Albert scale free network (BA) (2|| . Starting from a com- 
plete graph on uq vertices, one of the remaining K — no 
vertices is chosen. This vertex has m edges to attach 
as follows. With probability 1 — a, the vertex attaches 
an edge to an existing vertex with a probability propor- 
tional to the existing vertex's degree (i.e., by preferential 
attachment). With probability a, the edge is connected 
to any of the existing K — 1 network vertices with a fixed 
probability. This procedure is repeated m times, once 
for each edge. When a = 0, one obtains a BA network 
with a power law degree distribution, and when a — 1 
one obtains an ER random graph. Intermediate a give 
hybrid distributions with intermediate levels of hetero- 
geneity between the heterogeneous BA networks and the 
essentially homogeneous ER random networks. Networks 
are generated with K = 10 4 vertices, and average degree 
2m £ {4, 6, 8}. For each value of 2m, networks are gener- 
ated with a £ {0.00,0.10,0.20,0.40,0.60,0.80,1.00}. Fi- 
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FIG. 2: Simulation results for the evolutionary PD on networks with varying heterogeneity, and average degree 4 (a), 6 (b), 
and 8(c). The equilibrium level of cooperation is given as a temperature plot depending on both the network parameter Jl , 
and the cost-to-benefit ratio r. A point {x, y) in the plot, therefore, is colored according to the equilibrium cooperation level 
in the evolutionary game on a fixed network with heterogeneity given by x = « and PD cost-to-benefit ratio y — r. 



nally, each network is distilled down to its degree distri- 
bution by throwing away all other contact information, 
and a new uncorrelated network is reconfigured, consis- 
tent with that degree distribution, using the configura- 
tion model [2(| . The result is a maximally random net- 
work with the specified degree distribution. 

The point of this choice of networks is not any particu- 
lar topology, but rather, to give a range of varied topolo- 
gies with varied heterogeneity as measured by N ~ N V . The 
results of the simulations described above are consistent 
with previous work, and are summarized in Fig. [5]below. 

The theoretical prediction of equation [5] is compared 
with actual simulated dynamics on the networks above 
in Fig. 1 

The temperature plot in Fig. [2] shows that the data are 
in excellent agreement with the theoretical predictions, 
proving the effectiveness of the mean field parameters N 
and V, and validating the framework leading to Eq. ©. 

In particular, the black lines in each panel mark the 
predicted (by Eq. [6J r 50 , as a function of N ~^ v , where 
cooperators and defectors are expected to be at equal 
strength, and so, are predicted to be equally prevalent. 
Actual transitions from dominant cooperation (darker 
red) to dominant defection (darker blue) occur in the neu- 
tral tan colored regions between red and blue. Indeed, 



the black prediction line passes through the neutral, or 
nearly neutral, regions of the temperature plot. 

Statistical fluctuations are most prominent when V — 
4, while for V = 6 and V = 8, the predictions are ex- 
tremely accurate. This is not surprising in light of Eq. 

and the fact that when V = 4, values of N are small- 
est, and the term ^=1 most affects the value of r.50. 

We note that the mean field framework leading to 
Eq. ([S]) is extremely versatile, giving accurate predic- 
tions across networks with very different distributions, 
different levels of heterogeneity, and different average de- 
grees. Additionally, we have checked that the criterion 
also holds on random regular graphs with average degrees 
4, 6, and 8, where cooperation is virtually eliminated im- 
mediately, as predicted by Eq. ©. 

We turn to the assumption that simplifies Eq. 
namely, that the numbers of D-neighbors of the aver- 
age agents at either end of a C-D edge are comparable. 
While the accuracy of the predictions in Fig. [2] serves 
as a partial justification, we can consider the assumption 
directly in the simulations. Fig. [3] shows a histogram 
of the distribution of differences in the numbers of D- 
neighbors over all edges connecting a cooperator and a 
defector, normalized by N . The data was collected af- 
ter the ten-thousandth round of play and updating on a 
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FIG. 3: The actual probability distribution of , the dif- 
ference in the number of D-neighbors over all C-D edges. 
Simulations were run on a network with 5 x 10 3 vertices, 
and with average degree V = 6 and average neighbor degree 
N = 13.35. The data in the histogram was collected from 
simulations performed with r — 0.50, and equilibrium coop- 
eration level of 0.67 after 10 4 rounds of play. At r = 0.55, 
cooperation levels are below 0.50. 



of the individuals exceeds the cost-to-benefit ratio of the 
altruistic act. Genetic relatedness is measured by the 
probability that two genes randomly selected from each 
individual at the same locus are identical by descent pj. 
In the context of a social network, the parameter N ^ v 
serves as a natural definition for a notion of social relat- 
edness. Like genetic relatedness, social relatedness lies in 
the interval [0, 1), with larger values indicating increased 
relatedness. If two networks have the same fixed average 
agent size V, then there is more social cohesion in net- 
works with larger, more influential neighbors. As a re- 
sult , N emerges as the parameter governing social viscos- 
ity 
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28j , where larger neighbors increasingly facilitate 
relatedness, and through this, cooperation. Similar par- 
allels have been drawn before, particularly in 27] where a 
weak selection model was considered, but doesn't appear 
relevant to the case of strong selection considered here 
as it fails to distinguish between networks with different 
topologies, but the same average degree. 

In conclusion, we have shown that a simple analytical 
framework, using basic ideas form the theory of complex 
networks, can effectively predict the success of coopera- 
tion in an evolutionary PD on varied network topologies. 
Moreover, the analysis suggests a network-based evolu- 
tionary rule that nicely parallels Hamilton's classical ge- 
netic rule for kin selection. 



network with 10 4 vertices, average degree V = 6, average 
neighbor degree N = 13.35, and for r = 0.50. Note that 
N ~ v = 0.55, at which point cooperation levels are below 
50%. The distribution of is sharply peaked at 0. We 
then let the system run for another 10 3 rounds, comput- 
ing the average value of after each round. Finally, 
averaging over these 10 3 data points gives an overall av- 
erage value of ^ = -0.046. 

There is also a nice connection between the criterion of 
Eq. ([6]) and the results in [13] • In that paper the authors 
showed that the weighted (by the cost-to-benefir ratio r) 
average equilibrium cooperation level-call it y— on the 
network depended on the network parameter x = 4£ in 
a linear way. The regression line in that paper gave was 
given by y — — 1.0074x-|-.9322. Notice that the regression 
line is very close to y = —x + 1. Inserting x = one 
gets y — N Z , and the network parameter of Eq. ^ 
appears again. This seems to be a kind of mean value 
relationship, where the global average cooperation level 
taken over all values of the cost-to-benefit ratio also gives 
the local transition value ro.so- 

Finally, notice the similarity between the criterion of 
Eq. ((6|) and Hamilton's rule for kin selection. Despite 
arising in a completely separate context, Hamilton's rule 
gives a genetic criterion for the emergence of altruistic be- 
havior between individuals when the genetic relatedness 
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